Doctor: an Integrated so Ftware Fault Injec T Io N Envir Onment

This paper presents an integrateD sO ftware fault injeC T iO n enviR onment (DOCTOR) which is capable of injecting various types of faults with diierent options, automatically collecting performance and dependability data, and generating synthetic workloads under which system dependability is evaulated. A comprehensive graphical user interface is also provided. A special emphasis is given to the portability of this dependability experiment tool set. The fault-injection tool supports three types of faults: processor faults, memory faults, and communication faults. It also allows for injecting permanent, transient or intermittent faults. The proposed design methodology for DOCTOR has been implemented on a distributed real-time system called HARTS 1], and its capability is demonstrated through numerous experiments. Dependability measures, such as detection coverage & latency and the associated performance overhead, are evaluated through extensive experiments. Communication fault injection is used to evaluate a probabilistic distributed diagnosis algorithm. The results show that the algorithm performs better than its predicted worst case, yet is quite sensitive to various coverage and inter-processor test parameters. Any opinions, ndings, and conclusions or recommendations expressed in this paper are those of the authors and do not necessarily reeect the views of the funding agencies.